Color endoscope with charge coupled device and television viewing

ABSTRACT

A color endoscope having a component viewing head that is easily inserted into a relatively small opening or cavity and an electrical section that is capable of producing a readout signal containing full color image information that is compatible with standard format television for storing, displaying or otherwise processing the color information.

BACKGROUND OF THE INVENTION

This invention relates to an endoscope for processing color informationand providing a read-out signal that is fully compatible with anystandard format television equipment.

In U.S. Pat. No. 4,074,306, there is disclosed an endoscope system forproducing a full color image of the region scanned by the viewing headof the system. In this prior art device, the image information containedin the viewing region of the system is separated into the three primarycolors of red, green and blue, and the images then sequentially laiddown, one on top of the other, on a Braun tube to recreate the originalfull color image. The three primary color images are created by eithermechanically filtering the illuminating light brought into the cavityusing a single light source and a rotating filter disc or by breakingdown a reflected light image of the viewing region using a series ofdichroic mirrors.

Although the prior art device performs well, it nevertheless requiresspecial equipment to sequentially process the three independent read-outsignals. Mechanically driven filter discs also can present timing andbalancing problems which, under certain conditions, can be troublesome.Because the disc mounted filters move through an arcuate path of travel,a sharp, well defined edge between images, which is essential in a highspeed system, cannot be obtained. Lastly, the use of dichroic mirrorsand the like in the viewing head of the system increases the size of thehead and thus makes it difficult to insert into relatively small bodycavities or openings thereby limiting the usefulness of the instrument.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to improve endoscopesfor providing color pictures of a remote viewing region.

It is another object of the present invention to provide an endoscopesystem that is capable of utilizing standard format video equipment forstoring, displaying or otherwise processing color image information.

A still further object of the present invention is to improve theillumination system used in an endoscope for producing three primarycolor light images of the visual information found in the viewing regionof the instrument.

Another object of the invention is to eliminate the need for specialread-out equipment in a color endoscope system.

Yet another object of the present invention is to simplify the componentparts used in an endoscope system for providing a full color picture ofthe visual information contained in the object plane of the system.

A further object of the present invention is to produce an enlarged,high resolution, picture of the scene found in the viewing plane of anendoscope using standard video equipment.

These and other objects of the present invention are attained by meansof an endoscope having a compact viewing head which is easily insertableinto a relatively small body opening and an external electronics sectionthat is arranged to receive color image information from the viewinghead and provide a read-out signal that is fully compatible withstandard format television equipment. Three separate light images, eachcontaining data relating to an individual primary color, are generatedby electrically strobbing a series of lamps having rapid response times.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of these and other objects of the presentinvention, reference is had to the following detailed description of theinvention to be read in conjunction with the following drawings,wherein:

FIG. 1 is a block diagram of an endoscope system according to a firstembodiment of the invention wherein a full frame interlaced picture isproduced upon a standard television viewing screen;

FIG. 2 is a block diagram of a simplified version of the system shown inFIG. 1;

FIG. 3 is a partial view in section showing the illumination systemutilized in the present invention;

FIG. 4 is a view taken along lines 4--4 in FIG. 3 showing the end viewconfiguration of the viewing head employed in the present invention; and

FIG. 5 is a block diagram of an endoscope according to anotherembodiment of the invention.

FIG. 6 is a block diagram illustrating a still further embodiment of theinvention.

DESCRIPTION OF THE INVENTION

As shown in FIG. 1 of the accompanying drawings, the endoscope of thepresent invention, which is generally referenced 10, is made up of aviewing head 11, which is adapted for insertion into a relatively smallcavity or opening, and an electrical section 32 that is arranged toreceive visual data signals from the viewing head and convert the datainto an output signal that is fully compatible with standard formattelevision equipment for storing, displaying or transmitting the signalinformation. This type of equipment can include but is not limited tovideo tape recorders, television monitors, television receivers and thelike.

The present system also includes a light handling section 12 containingthree individual strobe lamps 13-15 that are sequenced in a programmedfiring order by means of a strobe drive 17. Positioned at the lightemitting surface of the lamps are optical elements 18-20 which serve tofilter the emitted light and to focus the light upon the light entranceface of a fiber optic bundle. Each lamp is a high intensity unit capableof producing white light without generating harmful amounts of infrared.The lamps can be adjusted over a relatively wide operating range withoutsacrificing color temperature. Each lamp, when pulsed on or off,exhibits a rise or fall time within the 10 to 100 microsecond range.

In practice, each of the optical elements 18-20 is specially prepared totransmit only light relating to one selected primary color whileblocking all other light. The present illumination system is programmedto strobe light into a light entrance face of the fiber bundle in a red,green and blue sequence although any desired firing order may beselected.

As seen in FIG. 3, the proximal end of the fiber bundle 22 istrifurcated with each arm being positioned adjacent to one of theoptical elements 18-20. The light entrance face of each arm is generallyperpendicular to the optical centerline of the adjacent element and liesabout within the focal place of the element. Accordingly, apreponderance of the light passing through each element is caused toenter the bundle and is channelled along the flexible bundle into theremote viewing head 11.

The bundle is bifurcated at the distal end thereof to provide for betterspace utilization in the viewing head and to reduce the shadowing effectnormally produced by a single bundle. The three color carrying sectionsof the bundle are further randomized at the distal end to producehomogeneous mixing of colors in the viewing region and thus provide foruniform illumination during each strobed imaging cycle. The illuminationreleased from the bundle is directed into the object or viewing plane ofthe system by means of a lens or lens system 26. The lens and the fiberbundle may be either physically or optically adjusted, or both, toproduce optimum illumination within the object plane.

Located immediately below the light carrying fiber bundle is anobjective or image forming lens 28. The objective is arranged to focusan image of the scene contained in the object plane thereof upon thelight receiving surface 29 of a self-scanning solid state imaging devicesuch as charge coupled device (CCD) 30 located in the image plane of theobjective. Although a simple lens system is depicted in the drawings, itshould be clear to one skilled in the art that a more complex opticalsystem can be herein employed without departing from the teachings ofthe present invention. Preferably, the objective lens system is providedwith a wide angle viewing accommodation wherein the view angle 0° isabout 85°.

As disclosed in further detail in the February 1974 issue of ScientificAmerican, a CCD imager contains a number of photosensitive pictureelements, generally referred to as "pixels", which are perpendicularlyaligned with reference to the optical centerline of the system to form agenerally rectangular grid pattern. In operation, light energy fallingupon the CCD receiving surface causes electrons contained within eachpixel region to accumulate or gather into charge packets. After a timedinterval the charge in each packet is applied to an electrode associatedwith the pixel thereby providing an electrical read-out of the visualimage information recorded upon the receiving surface. The data storedin the imager is then clocked out of the element in a line by linesequence similar to that employed in a serial flow shift register. Theimager employed in the present invention has a driver-amplifier 33 (FIG.3) that is operatively associated therewith and which serves both as ameans for driving the CCD components and for providing immediatepreamplification of the read-out data. Preamplification eliminatesunwanted generation of noise or crosstalk in the lines and the driverreduces the number of electrical leads needed in read-out line 31connecting the imager with external electrical package 32.

With further reference to FIG. 4, the viewing head includes a pair ofillumination windows 34--34 and a single image viewing window 35. Asnoted, the bifurcated distal end of fiber bundle 22 provides added spacewithin the viewing head whereby components, such as the biopsy device 36illustrated in FIG. 4, may be conveniently located therein. Means towash fluids and the like from the windows may also be located in thisregion along with, or in place of, the biopsy device.

Referring now more specifically to FIG. 1, the electrical section 32 ofthe system is arranged to accept the visual information signal from thesolid state CCD imager and place the information in a format that iscompatible with standard video processing equipment. As is typical inmost, if not all, standard format television equipment, each pictureframe is made up of two interlaced fields of data which combine toprovide a faithful rendition of the region viewed. Each field furthercontains a preselected number of horizontal data lines which are laiddown on the viewing screen within a prescribed period of time. Althoughthe number of data lines and the duration of the field periods, mayvary, the operation of the video systems remain basically the same.

For explanatory purposes, the apparatus of the present invention will bedescribed in reference to a television format in which each fieldcontains 244 horizontal lines of data which are presented in 1/60 of asecond. Accordingly, each frame will contain 488 lines of data and take1/30 of a second to complete. To accommodate this format, the CCD imageris provided with a 488×358 pixel grid pattern. The horizontal number of358 pixels can also be varied depending upon the bandwidth of thereceiver and the degree of resolution desired.

In the imaging system shown in FIG. 1, the first field period of 1/60 ofa second is utilized to load visual color information supplied by theCCD into three memory units on registers 40, 41 and 42. The threememories are located in a first memory bank 43. In operation, each lampin the illumination system is triggered once during each field periodthrough means of a strobe drive 17 acting in response to a timing signalfrom the clock of master timing circuit 45. The master timer may consistof synchronization generator which can be used directly or with logicgates to provide all the switching functions required in the presentapparatus. Typically, each lamp is flashed on and off at 1/180 of asecond interval whereby the region in the object plane of the viewinglens is illuminated in an ordered red, green, blue sequence during eachfield period.

During the first red imaging interval, the CCD imager accepts red visualimage information and converts it to an electrical output data signalthat is applied to the preamplification section of the driver-amplifier33 and then passed to video amplifier 38. The interval from illuminationto read-out takes 1/180 of a second. At the start of the red imagingcycle, the timing circuit has also conditioned analog switches 46 and 47to be positioned so that the data signal from the amplifier is clockedinto a first memory unit 40. In practice, each unit is preferably ananalog CCD chip, however, any suitable shift register for storing thistype of data may be used. As will become apparent from the disclosurebelow, each CCD memory in this particular embodiment is required tostore data only during one of the two fields making up a frame andtherefore the memory unit utilizes a simplified 244×358 pixel gridarrangement.

On the following green imaging cycle, the CCD imager is clear of dataand is placed in a condition to accept green color information. Hereagain, the output data signal from the imager is amplified and shiftedin a line by line sequence into the first memory 40 in the bank. Thisaction in turn causes the red image data contained in memory 40 to passserially into memory 41. Similarly on the next blue imaging cycle, bluecolor information that is generated is shifted into memory 40 whereuponred data passes serially into memory 42 and green data passes intomemory 41. This completes the first or odd field period.

At the end of te first field, the master timing circuit causes analogswitches 46-48 and 56,57 to be repositioned whereby color informationstored in memories 40-42 is simultaneously clocked into the videoprocessor at video-speed. The parallel flow of information issynchronized with the 244 line presentation of the second field by meansof the master timer. As this data is being clocked into the videoequipment, new data is being fed into a second bank of memories 44 fromthe CCD imager.

In the video processor 50, the initial three color data is corrected andplaced in a form that is acceptable by standard television equipment. Asis well known in the art, the processor includes a transmitter matrixfor combining the three color signals into composite luminance andchrominance signals that are used in the receiver to control the variouspicture functions. To obtain instantaneous viewing of the data, thesignal is modulated at unit 51 and passed directly to a T.V. receiver 52for providing a visual presentation thereof. The modulator can be anyone of many such commercially available units such as a radio frequencymodulator that is compatible with all standard NTSC televisiontransmitter matrixes and receivers.

During the second or even field period, the lamps are again strobbed inthe noted sequence and the primary color information shifted seriallyinto registers 55, 54, 53 as noted above in a red, green and bluesequence. Upon receipt of the new data, the analog switches are againrepositioned through means of timing circuit 45. The stored data ispassed in parallel flow into the processor 50. A second field of colorinformation is then processed and is interlaced within the first fieldto create a high resolution color rendition of the viewing region whichcan be presented upon the screen of receiver 52, while an odd field isbeing stored in the memory bank 44.

FIG. 2 illustrates a simplified arrangement of the invention wherein thenumber of memory units needed to create a color picture is reduced. Thevideo processing and viewing equipment 50-52 again is standard formatequipment and the illumination system and viewing head remain the samewith the like components being referenced with like numerals. Hereagain, a full screen presentation is furnished. However, because of thereduction in memory units, one field in every frame must be blanked.This causes a reduction in the resolution of the picture but enables asimpler 244×179 pixel format to be employed in regard to the CCD imager.Correspondingly, only two memory units 60 and 61 are required and thesememories can be greatly simplified and thus more economical to build.

At the beginning of the first field, analog switches 63, 64 arepositioned as shown so that red and green image information clocked outof the CCD imager is shifted serially into memories 61 and 60respectively. Through means of the master timer 62, each color signal isclocked out of the imager during a 1/120 of a second interval. At thestart of the next field, the read-out speed of the CCD imager is changedvia the master timer. Switches 63 and 64 are also repositioned so as tofeed blue data stored in the imager 30, along with the data stored inthe memories 60,61 in parallel flow relationship into the processor 50,through the modulator 51 and finally present the data upon TV receiver52 as explained above.

As can be seen, in this simplified embodiment of the invention, only onefield of each frame is employed to generate a visual display at the TVscreen. The second field of the frame, of course, is blanked or groundedout during the period when new data is being stored in the memory units.As a result, the resolution of the picture may be slightly affected dueto the lack of interlacing. However, the detail and quality of thepicture is more than satisfactory to enable the instrument to be usedfor its intended purpose.

The circuitry required to produce the color picture is simplified andthe cost of the imager and memory units considerably reduced through theuse of the 244 line arrangement.

Turning now to FIG. 5 there is shown in block diagram form a furtherembodiment of the invention also utilizing the previously describedviewing head and illumination system. Here again, like numerals are usedto depict similar or like parts. In this embodiment the output signal ofthe CCD imager is applied to an analog-to-digital (A-D) converter 65whose digital output is transmitted to a computer section generallyreferenced 67. In the A-D converter, color image information clocked outof the CCD imager is placed in six bit digital form. It has been foundthat a six bit output will contain sufficient information whereby a highresolution picture signal is provided to the video equipment. It shouldbe clear, however, that more or less bits of data can be similarlyemployed without departing from the teachings of the present invention.The A-D converter output is applied to muliplexer 71 and the data ismultiplexed onto one of the six output data lines in response to asignal from master timer 73. During the first field, three color data isloaded into a first bank of memory units 74 while during the secondfield three color data is loaded into a second bank 75, also containingthree memory units.

As noted, the CCD imager will preferably have a 488×358 pixel gridarrangement for delivering a full screen interlaced picture at thereceiver 52. The lamps 13-15 are thus strobbed at 1/180 of a secondintervals to provide one full color field every 1/60 of a second. Afterthe color signals are converted to digital form, they are multiplexedinto the two banks of memories 74, 75, each of which contains a red,green and blue image retention section.

During the first field, the information stored in bank 74 is passed tooutput multiplexer 76 and then on to three digital-to-analog converters77, 78 and 79. In response to a signal from the master timer, the threecolor signals are simultaneously passed from the D-A converter on to thevideo processor 50, modulator 51, and video receiver 52 at video speed.During the first field, new color data is being loaded into the red,green and blue memories of the lower bank 75. At the start of the secondfield the function of the memories is reversed, thereby enabling thesystem to deliver color data to the video equipment during each field ofa frame.

As is well known in the art, an endoscope is typically employed to viewthe inside of body cavities. It is therefore essential that the viewinghead, that is, the portion of the device that is inserted into the bodycavity be as small as possible to provide for the safety and the comfortof the subject. As should be apparent from the instant disclosure,because the present device utilizes a single CCD imager in the viewinghead, its size can be minimized without sacrificing performance. Itshould be further noted that the size of the CCD imager can be furtherreduced by changing the pixel grid arrangement to something less thanthe 488 line format used in standard TV equipment.

For example, a system using a 244×134 grid pattern may be employed toprovide interlacing as disclosed in the inventive embodiment of FIG. 1.In this case, the total area of the picture on the screen is reduced.Although the picture is slightly reduced in size the resolution of thepicture, however, is relatively unaffected. Furthermore, in this reducedpicture format, the unused portion of the screen may be electronicallyimaged to present added information relating to the subject being viewedor any other related data that might be desired.

Because the present apparatus is compatible with all standard videoequipment, the subject matter being viewed may be video taped forstorage and future reference. This can be simply accomplished byapplying the output of modulator 51 to any standard video processor 80as shown in FIG. 5. Similarly, as shown in FIG. 2, raw data taken fromthe video processor can be sent directly to a standard video-monitor 59for viewing, thus eliminating the need for video units 51 and 52.

Turning now to FIG. 6, there is shown another embodiment of theinvention wherein color data that is stored in a series of memories iscontinually fed to the video equipment at video speed to provide a fullframe, fully interlaced color signal suitable for viewing, tape storageor the like. In this embodiment, however, each primary color signal thatis stored in one of the memory units is periodically updated with theup-grading being accomplished at a speed that is considerably slowerthan video speed. This result is herein achieved using four memory units85-88. Three of the units feed data to the video equipment while thefourth is being upgraded.

Here again, the remote viewing region is illuminated by light of aprimary color through means of the three lamp system as previouslydescribed. The light image is used to expose the CCD 30 and anelectrical read-out signal indicative thereof is clocked out of thedevice in a line-by-line sequence. As shown in FIG. 6, the read-out datafrom the imager can be selectively applied to one of the four memoryunits. In operation, each of the memories is wired into the system inthe same manner. The input signal data is passed from the amplifier 38into each memory via electrically operated switch S-1. A portion of theoutput signal is divided out, amplified, and then fed back to the inputof the memory via line 91 and electrically operated switch S-3.

In practice, red, green and blue data from the CCD imager issequentially stored in three of the memories, as for example, memories86, 87 and 88, by sequentially cycling the associated input switches S-1in response to a preprogrammed signal from master timer 93. With thecolor information stored in each unit, the S-2 and S-3 switchesassociated therewith are simultaneously closed at the beginning of aframe and the stored information is clocked out of the units in parallelflow at video speed.

As the information is being fed out of each unit in a line-by-linesequence, the old data is also being restored in the unit through meansof the feed back network. By this means, color image data is continuallyprovided to the multiplexer during one field of each frame while, at thesame time, the memories are isolated from the CCD imager.

While data concerning the three primary colors is being forwarded to themultiplexer at video speed, updated data concerning one of the primarycolors can be passed from the imager into the fourth memory 85 at arelatively slower speed. During the upgrading period switch S-1 of unit85 is closed while S-2 and S-3 are opened. Again positioning of theswitches is accomplished through means of the timing circuit 93. Whenmemory 85 has been supplied with the desired 244 lines of updatedinformation, the memory containing old data relating to the same primarycolor is taken off the line by opening associated switches S-2 and S-3in an ordered timed sequence. Initially S-3 is opened while S-2 is heldclosed to permit the stored data to be cleared into the multiplexer 90.Once the old data is cleared, switches S-2 and S-3 associated with theupgraded memory are simultaneously closed in timed relationship with thebeginning of the next field period whereby the new data is forwardedfrom the upgraded memory in parallel flow with the remaining stored datainto the multiplexer 90. Switch S-1 of the upgraded memory, at thistime, is now opened while that associated with the cleared memory isclosed.

The memory that has been cleared is now in a condition to receive new orupdated data relating to a second primary color whereby the above notedsequence of events is repeated. Each color is thus sequentially updatedto continually upgrade the picture information.

The three color data passed into multiplexer 90 is processed, as is wellknown in the art, and multiplexed out by means of three output datalines. The multiplexed signal is passed on to video processor 50,modulator 51 and viewer 52.

While this invention has been explained with reference to the structuredisclosed herein, it is not confined to the details as set forth andthis application is intended to cover any modifications or changes asmay come within the scope of the following claims.

We claim:
 1. An endoscope for providing color information in a form thatis capable of being displayed on a standard format video receiverutilizing odd and even interlaced fields for each frame of informationdisplayed, the endoscope includinga viewing head that is capable ofbeing inserted into a body cavity that contains a self-scanning solidstate imaging device for recording light images of objects positioned inthe viewing region of the head and providing a line by line electricalread-out signal containing the recorded information, lighting means forbringing illumination from a remote location into the viewing region ofsaid viewing head to sequentially light the viewing region with light ofprimary colors during each field related period so that read-out signalsrelating to the color images recorded by the imaging device are clockedout of the device in a sequential flow relationship, a first memory unitfor individually storing read-out signals regarding each of the primarycolors clocked out of the imaging device for display during an evenfield period, a second memory unit for individually storing read-outsignals regarding each of the primary colors clocked out of the imagingdevice for display during an odd field period, and switching means forforwarding the signals stored in said first memory unit simultaneouslyto a transmitter matrix for .[.combining the color information intocomposite luminance and chrominance signals for.]. display in a.[.standard format.]. television receiver during an even field periodand for forwarding signals stored in said second memory unitsimultaneously to said matrix for display during an odd field period. 2.The endoscope of claim 1 that further includes means for periodicallyupdating the information stored in each memory unit.
 3. The endoscope ofclaim 1 wherein said lighting means further includes an individual lightsource for each primary color with the light sources being remotelypositioned in regard to the viewing head and having flexible lighttransmitting means for bringing light from each source to the viewinghead.
 4. The endoscope of claim 3 wherein said light transmitting meansincludes a fiber optics bundle that is divided at the light input endinto a plurality of light input branches that are equal in number to thenumber of light sources with each branch being arranged to deliver lightfrom one of said sources into the bundle.
 5. The endoscope of claim 4further including timing means for activating each light source in anordered sequence.
 6. An endoscope for providing a full color videodisplay includinga viewing head that is capable of being inserted into abody cavity, the head containing a charge coupled device (CCD) forrecording visual information of objects situated in the viewing regionof the head and, in response thereto, providing a line by lineelectrical output signal containing the recorded information, aplurality of light sources, each of which is capable of producing lightof a primary color, said light sources being located at a location thatis remote from the viewing head whereby the light sources remain outsideof the body cavity. a randomly dispersed fiber optic bundle that isarranged to bring the illumination from each source into the viewinghead to illuminate the viewing region, activating means for illuminatingeach lamp in an ordered sequence whereby the viewing region isilluminated with light of each primary color, memory means beingarranged to accept the output signals of the charged coupled device andto individually store the signals relating to each of the primarycolors, and switching means for periodically forwarding the signalinformation stored in the memory means simultaneously to a transmittermatrix for combining the color information signals into compositeluminance and chrominance signals that are compatible with a standardformat television receiver.
 7. The endoscope 6 wherein said memory meanshas a first section for storing information for display during an evenfield period and a second memory for storing information for displayduring an odd field period.
 8. The endoscope of claim 7 furtherincluding means for periodically updating the information contained ineach memory section.